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ABSTRACT 

A method of evaluating the Secondary School 
Mathematics Curriculum Improvement Study (SSMCIS) now in its fourth 
year at Teachers College, Columbia University, is discussed. The main 
task of SSMCIS is the production and tryout of textbook materials, 
although teacher training is also an important component. Informal 
teacher feedback and sporadic testing provide the main thrust of 
evaluation of this program to date. One study notes the discrepancy 
between the views of teachers and students on some features of the 
text materials and suggests the need to acquire more direct 
information from the students concerning course materials than has 
previously been the case. Most of the evaluation activities have 
concentrated cn support functions rather than the project itself. The 
question of whether full-scale formative evaluation of the study 
would be any more effective in influencing curriculum revision is 
raised in the conclusion. (AE) 
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Despite the widely-held view that the past decade or 
so has seen a revolution in our school mathematics curricu- 
lum, a convincing argument can be made that to date the 
reform has been chiefly an up-dating of the traditional 
curriculum. The big breakthrough--a restructuring of the 
curriculum that would abandon the separation of mathematics 
Into compartments labeled "arithmetic," "algebra," and "geo- 
metry" --ha 8 yet to occur. Since the turn of the century, 
educators have been calling for a school mathematics program 
that would be organized around the fundamental concepts and 
structures of matheuatlcs--concepts such as sets, relations, 
mappings, and functions! structures such as groups, fields, 
and vector spaces. Yet the traditional course boundaries 
have proved remarkably impervious to change. 

One attempt to produce a unified mathematics curriculum 
for grades 7 to 1? is tho Secondary School Mathematics 
Curriculum Improvement Study (or SSMCIS), located at Teachers 
College, Columbia University, directed by Howard P. Fehr, 
and now in its fourth year, influenced by work in this 
country such as the Report of the Cambridge Conference on 
School Mathematics and by European attempts to reconstruct 
the mathematics curriculum, the SSMCIS is developing a 
curriculum for college-capable 8tudent8--specifieally the up- 
per 15 to 20 percent of the population in mathematical 
ability. The goal is to use the unifying concepts of mathe- 
matics as a more efficient basis for organizing the curricu- 
lum so that much that has been considered undergraduate 
mathematics con be introduced into the high school program. 
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The main task of the Study has been the production and 
tryout of text materials, although teacher training has 
been an important component. The typical mode of operation 
for the textbook writing has been as follows. After a panel 
of consultants (mathematicians and creators) has set up the 
brood outlines for a course-*in line with the overall sylla- 
bus plan adopted at the outset of the Study in 19^6--a con- 
ference of writers and consultants is held in June to hammer 
out the details of each new course, as well as to suggest 
revisions of material previously written. A preliminary 
version of each new course is written during the remainder 
of the summer, tried out during the year in a half dozen 
or so classes in the Metropolitan New York area with two 
teachers per class, and then revised the following summer 
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In the light of Information from tests, observations, and 
reports from the teachers. The teachers receive special 
training each summer at Teachers College, and in addition, 
use their previous year's experience to assist In preparing 
teacher's commentaries for the courses undergoing revision. 

At present the experimental version of Course IV 
(tenth grade) Is being tried out in five classes. Pinal 
versions of Courses I and II (grades 7 and 8) are avail- 
able now* the revision of Course III will be ready in the 
fall, Even though most teachers need special preparation 
to teach the materials--receiving it chiefly through summer 
institutes at Teachers College, the University of Maryland, 
and the University of Arizona--the number of classes using 
the materials has risen markedly each year, so that currently 
there are obout 150 classes using Course I and 70 classes 
using Course II. 

The accelerating rate of adoption of those challenging 
and unusual materials suggests that SSMCIS must be doing 
something right, but of course one of the functions of e- 
valuation is to check the nature and validity of this "some- 
thing that seems to be so right. 

I would like to take the rest of my time here today, 
therefore, to sketch some of the problems and prospects of 
evaluating the SSMCIS program. Those of you who would like 
further details on the nature of the SGMCIS curriculum may 
write Professor Howard Fehr, Director of SSMCIS, at Teachers 
College and request the current information bulletin. Also, 
an article on the SSMCIS by Pehr and James Fey appears in 
the December 1$<$9 issue of the American Mathematical Monthly . 

I think it's fair to say that various forms of "teacher 
feedback" hAve been the major source of evaluation used by 
SSMCIS in revising its material*. One of the advantages 
of a small project, with tryout centers located nearby, is 
that the teachers of the experimental classes become an in- 
tegral part of the project, providing a sort of continuous 
monitoring of the curriculum’s effectiveness. Our staff 
knows the teachers personally, can visit their classes 
frequently, and can respond at any time to questions or re- 
quests for assistance. In addition to their work with the 
writing teams during the aumaer, the teachers of the experi- 
mental classes convene at Teachers College several times 
during the year for an all-day Saturday meeting, when progress 
can be assessed and common problems discussed. 

Student performance on tests constructed by the staff 
does, of course, provide information about the materials. 

To date, however, the test9 have been used more to indicate 
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student 1 s progress to teachers, administrators, parents, 
end the students themselves than to diagnose weak spots 
in the materials. We are currently trying to make our tests 
more diagnostic by being a little more careful and syste- 
matic in constructing Items to fit the taxonomy of objectives 
devised at the outset of the study. 

Several research studies done In the Department of 
Mathematical Education at Teachers College have treated 
various facets of the SSMCIfl program. The one that deals 
mo3t directly with questions of program evaluation has 
just been completed. It Is the doctoral disserts don of 
Brother Michael Hoban, and it is a kind of intrinsic, non- 
comparative evaluation study. Brother Hoban took one chapter 
from Course Il--the eighth grade course — dealing with trans- 
formation geometry and attempted to judge the effectiveness 
of the chapter (and the teachers) in meeting the objectives 
laid out for it, A four-man jury of mathematicians and 
mathematics educators associated v?ith the project judged the 
evaluation blueprint Hoban devised and test items ho wrote. 
Items that survived this screening were pilot tested in 
parallel forms to determine the necessity for a pretest and 
to eliminate items that failed to discriminate. 

In the main study, 9 classes studied the chapter and 
then took a postte3t comprised of items that survived the 
screening and the pilot testing. Vhile the chapter was being 
taught, observations were made of the classrooms, and after 
the testing, questionnaires were administered to students 
and teachers. 

Because Hoban used a combination of results from the 
pilot test, the juror 1 s ratings of item difficulty, and 
his own judgment in setting performance standards for the 
test, the test cannot, strictly speaking, he called "cri- 
terion-referenced." And because he was unable to come up 
with a workable scheme for rating classroom behavior, he lost 
a good opportunity to study how class achievement might have 
been related to the teacher's handling of material. 

Nonetheless, the study does suggest that the materials 
are relatively successful--most of the classes met the 
standards set for them on most of the items. Moreover, there 
were indications that both teachers and students found the 
« ideas of transformation geometry treated in the chapter to 
be interesting and enjoyable to study. 

Perhaps the most important finding was the uncovering 
cf a discrepancy betv»een the vlevis of teachers and students 
about some features of the text materials. Whereas the 
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tenchers considered the reading level and the way new terms 
and definitions were introduced to hove been appropriate, 
students claimed they had some difficulty in reading the 
chapter and in understanding all of the new terminology. 

On the other hand, there were some concepts ( translation 
and rotation, for instance) that the students cTaTmed they 
understood but that the teachers weren' t sure the students 
had completely grasped. And in general, the students were 
more concerned than teachers apparently thought they were 
about the "need" for studying transformation geometry and 
its practical value. 

This suggests that we in the project should begin to 
get more direct information from the students about their 
reactions to the materials, to supplement the teachers' 
views. After all, these students are bright and articulate. 
Though they may lack the perspective on the curriculum that 
a teacher might have, they should be reasonably competent 
critics of what they are learning and reasonably able to 
convey their criticisms to us, 

In the S5MCIS project many cf our "evaluation" acti- 
vities have a support function--f or instance, the pro- 
vision of midyear* and final examinations for each course, 
the development of special Regents examinations for our 
classes in New York State, and perhaps eventually the develop- 
ment of alternative college entrance examinations. These 
activities, important though they may be, take up the bulk 
of the limited time and monpovrev we con give to evaluation. 

I don't see much prospect of our being able to do much more 
in the way of formative evaluation than we ore already doing. 

At the risk of claiming virtue for necessity, then, 
let me raise the question of whether we should attempt to 
do more even if ue could, t'hat I mean is" this: it has been 
my experience in this project and in others that when writing 
textbooks for school children, mathematicians are guided 
more by their own internal vision of how things ought to be 
than by anything else. Rational ax'guments from their col- 
leagues can roll off their backs like water. Let me leave 
the question for you to ponder, as I have would a full-scale 
formative evaluation be any more effective in influencing 
curriculum revision in a project like ours than the informal 
teacher-feedback/sporadlc-testlng model used to date has 
been? I really wonder if it would be worth the effort to do, 
even if we could. 




